Web Survey Bibliography
Relevance & Research question
Mobile phones are replacing the PC as key devices in social science data collection. In daily life, mobile phones are used for short interactions. Successful data collection strategies over mobile phones should therefore also be brief for respondents.
Questionnaires for attitude research are often very long. We argue that there is a trade-off to be made. Should questionnaires on mobile devices remain long, risking dropout, or should such questionnaires be split up (from here on called chunks) to optimize data quality?
Methods and data
We report on an experiment conducted in the probability-based LISS panel in the Netherlands, carried out in December 2015. We used a ‘within’ design of data chunking. Panelists who owned a mobile phone with Internet connection were randomly assigned to either:
a) The normal survey (about 20 min)
b) The same survey cut into three chunks, with each chunk offered after a week
c) The same survey cut into ten chunks, with each chuck offered every other day.
Results
First, we investigated the number of complete and incomplete responses and looked at indicators for data quality (straightlining, primacy effects, survey length). We find that more respondents are completing the questionnaire when it is offered in chunks (condition b, and especially c), but also that chunking results in more item missings. We find little evidence for effects on data quality.
Finally, we report on the differences we find in the factor structure when the questionnaire was split into chunks, or was completed in one go.
Added value
The idea of data chunking is not new. ‘Planned-missingness’ designs have been implemented in web surveys successfully in the past. This study is however the first to study data chunking in the setting of mobile phone surveys. We believe that more and more data will be collected using mobile phones (already 5-25% of all web surveys are taken on mobile phones), and that understanding how to design questionnaires for mobile phones is of vital importance to both survey researchers, market researchers, and anyone using such data for substantive reasons in the future.
Web survey bibliography (81)
- Data chunking for mobile web: effects on data quality; 2017; Lugtig, P. J.; Toepoel, V.
- Do Initial Respondents Differ From Callback Respondents? Lessons From a Mobile CATI Survey; 2016; Vicente, P.; Marques, C.
- Mobile Device Use in Web Surveys Among College Students: Predictors and Consequences for Data Quality...; 2016; Beach, S.; Musa, D.; Strotmeyer, S.; Schlarb, J.
- Web surveys for offline rural communities ; 2016; Gichohi, B. W.
- An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones...; 2016; Revilla, M.; Toninelli, D.; Ochoa, C.
- Pre-Survey Text Messages (SMS) Improve Participation Rate in an Australian Mobile Telephone Survey:...; 2016; Dal Grande, E.; Chittleborough, C. R.; Campostrini, S.; Dollard, M.; Taylor, A. W.
- When Should I Call You? An Analysis of Differences in Demographics and Responses According to Respondents...; 2016; Vicente, P.; Lopes, I.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Mobile Research Methods: Opportunities and challenges of mobile research methodologies. ; 2015; Toninelli, D. (Ed.); Pinter, R.; de Pedraza, P.
- Collecting Health Research Data - Comparing Mobile Phone-assisted Personal Interviewing to Paper-and...; 2015; van Heerden, A. C.; Norris, S. A.; Tollman, S. M.; Richter, L. M.
- The Effects of Questionnaire Completion Using Mobile Devices on Data Quality. Evidence from a Probability...; 2015; Bosnjak, M.; Struminskaya, B.; Weyandt, K.
- Designing web surveys for the multi-device internet; 2015; de Bruijne, M.
- Mobility Enabled: Effects of Mobile Devices on Survey Response and Substantive Measures; 2015; Barlas, F. M.; Randall, T. K.
- Innovations in Email Invitation Design for Today’s Digital World; 2015; Saunders, T.; Kessler, A.
- Emerging Technologies: The Rise of Mobile Devices: From Smartphones to Smart Surveys; 2015; Buskirk, T. D.
- Open narrative questions in PC and smartphones: is the device playing a role?; 2015; Revilla, M.; Ochoa, C.
- Mixed-method feasibility study comparing the outpatient assessment of burn patients using a tablet device...; 2015; Mitchell, S. S.
- Mobile Devices for the Collection of Sensitive Information; 2015; Maitland, A.; Mercer, A. W.; Tourangeau, K.; Williams, Do.
- The Impact of Mixing Modes on Reliability in Longitudinal Studies; 2014; Cernat, A.
- Global market research 2013; 2013
- Australia: building a 21st century readership survey; 2013; Green, A., White, H.
- The new swiss national readership survey: fit for the future ; 2013; Amschler, H., Hoffmann, J.
- Relative Mode Effects on Data Quality in Mixed-Mode Surveys by an Instrumental Variable; 2013; Vannieuwenhuyze, J. T. A., Revilla, M.
- Is the Sky Falling? New Technology, Changing Media, and the Future of Surveys; 2013; Couper, M. P.
- A report on the Confirmit Market Research Software Survey 2013; 2013; Macer, T., Wilson, S.
- Survey Breakoffs in a Computer-Assisted Telephone Interview; 2013; McGonagle, K.
- Impact of mode design on reliability in longitudinal data; 2013; Cernat, A.
- Mobility and Smartphones: a pilot study of travel data collection among experienced and inexperienced...; 2013; Douhou, S., Scherpenzeel, A.
- Mobile devices a way to recruit hard-to-reach groups? Results from a pilot study comparing desk top...; 2013; Toepoel, V., Lugtig, P. J.
- Mode effects in Labour Force Surveys - do they really matter?; 2013; Koerner, T.
- Comparison of quality of web survey and CATI data using unobtrusive response latencies; 2013; Mayerl, J.
- Comparability of substantive results between modes and incentive conditions in a probability-based telephone...; 2013; Pekari, N.
- Data Collection Method Comparisons for the 2011 Fishing, Hunting, and Wildlife-Associated Recreation...; 2013; Herbstritt, M., Hornick, D.
- The comparability of Don't Know answers between CATI and CAWI modes; 2013; Pohjanpaa, K., Jarvensivu, M.
- Measuring the same concepts in several modes in the "BIBB/BAuA-Employee-Survey 2011/12" ; 2013; Gensicke, M., Tschersich, N., Hartmann, J.
- Error Prevention through Interviewer Emulation? Introducing questionnaire dialogues in the Norwegian...; 2013; Gravem, D. F.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Comparing Tablet, Computer, and Smartphone Survey Administrations; 2013; Wells, T., Bailey, J., Link, M. W.
- Cross-Platform Measurement: User Experience With a Smartphone and Web Self- Reported Data Collection...; 2013; Petras, A. P., Duan, S., Dan, O.
- Examining the Feasibility of SMS as a Contact Mode for a College Student Survey; 2013; Crawford, S. D., McClain, C., O'Brien, S., Nelson, T. F.
- Surveys on Mobile Devices: Opportunities and Challenges; 2013; Couper, M. P.
- Benefits of Modular Design for Mobile and Online Surveys; 2012; Kelly, F., Johnson, A., Stevens, S.
- Unintentional mobile respondents; 2012; Peterson, G.
- Understanding Mode Effects between Mobile Web and Mobile SMS Surveys; 2012; Poduska, B., Johnson, E. P.
- Measures of Data Quality Across the RDD Frames; 2012; Lavrakas, P. J.
- Mobile Survey Participation Rates in Commercial Market Research: A Meta-Analysis; 2012; Bosnjak, M., Poggio, T., Becker, K. R., Funke, F., Wachenfeld, A., Fischer, B.
- Qualitatively Speaking: Mobile qualitative finally hits its stride; 2012; Bryson, J.
- Modular Survey Design for Mobile Devices; 2012; Johnson, A., Kelly, F.
- Using SMS Text Messaging To Collect Time Use Data; 2012; Brenner, P., DeLamater, J.
- Telephone Status, Attitudes toward Participation in Future Surveys, and Willingness to Join a Local...; 2012; Beach, S., Musa, D.